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ABSTRACT 

The response rates of university graduates and the 
cost per return were studied for a 20-item questionnaire presented in 
3 formats as follows: (1) a 2-page questionnaire with an accompanying 
self-addressed stamped envelope; (2) a condensed format with smaller 
type, on 1 page, with a self-addressed stamped envelope; and (3) the 
single-page questionnaire incorporated as a self-mailer. Nineteen 
questionnaire items were based on a 4-point Likert scale, and the 
20th item was an open-ended question scored on another 4-point scale. 
A wide-scale mailing to college graduates was made to survey their 
perceptions of the effectiveness of their undergraduate education. Of 
the 7,078 surveys sent out, 1,830 were returned (25.7 percent 
response rate overall). Response rates for the questionnaires were 
27.9 percent for the 2-page with return envelope, 26.3 percent for 
the single page with envelope, and 23.3 percent for the single page 
self-mailer. Responses were also similar across the eight university 
colleges. The most efficient format was the 1-page questionnaire with 
a return envelope (1.256 dollar per return) and the most expensive 
was the two-page with envelope (1.591 dollar per return). Although 
differences in the return rates for the three formats were 
statistically significant, there may be little practical impact to 
small differences in surveys with low return rates. Three tables 
present study findings. (SLD) 
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Survey Questionnaire Format Effect on 
Response Rate and Cost per Return 

A high percentage response rate is a major concern in a survey investigation 
using the mailed questionnaire. Although a sample may represent a well defined 
population, the analysis is limited to those who return the questionnaire. Higher 
response rates tend to enhance the validity of the sample as being more 
representative of the population. Unfortunately, low return rates are not uncommon 
for follow-up studies of university graduates (Smith & Bers, 1987). To increase the 
chance of graduates returning follow-up questionnaires, researchers have examined 
factors such as the appearance of the questionnaire (Boser, 1990), the length of the 
questionnaire (Harvey, 1988), and sponsorship and postage (Armstrong & Lusk, 1987; 
Fox, Crask & Kim, 1988). 

After reviewing research on mailed questionnaire response rates, 
Baumgartner and Heberlein (1984) noted the need for research on the effects of a wide 
range of questionnaire lengths on response rate. Although earlier studies had found 
higher response rates for stapled sheets versus single page questionnaires, no 
significant difference was found in a more recent investigation (Harvey, 1988). In a 
related investigation, Boser (1990) found no significant difference in response rates 
for stapled pages versus a booklet style questionnaire. 

Many questionnaire formats are used to gather survey information using the 
mailed questionnaire. Some use smaller type to present questionnaire items on a 
single page. Some save money by using a self-mailer in lieu of the return envelope. 
To what extent do these questionnaire formats enhance a higher response rate? Does 
the use of a self-mailer generate lower cost per returned questionnaire, thus 
producing a more efficient study? 

Purpose of the Study 

The purpose of this investigation was to contrast the response rates of 
university graduates and the cost per return for a 20-item questionnaire presented in 
three different formats: (1) a two-page questionnaire, returned in an accompanying 
self-addressed stamped envelope; (2) a condensed format using smaller type with 
questionnaire items on a single page, returned in an accompanying self-addressed 
stamped envelope; and (3) the single-page questionnaire incorporated as a self-mailer 
in lieu of the return envelope. 

Method 

A wide-scale questionnaire mailing to university baccalaureate graduates 
was made to survey their perceptions on the effectiveness of their undergraduate 
education. Graduates were asked to respond to 19 questionnaire items on a four-point 
Likert scale (l=Excellent: A model of good practice; 2= Good: Very favorable; 3=Fain 
Not good, not bad; 4=Poor: Unfavorable). The twentieth questionnaire item was an 
open-ended question, scored on the original four-point scale. Selection of the 
questionnaire format, style, directions and mailing procedures were consistent with 
desirable and validated questionnaire characteristics (Boser & Clark, 1992). 
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The mailing was randomly divided into three samples: (1) those sent the two* 
page questionnaire accompanied by a stamped, addressed return envelope; (2) those 
sent the one-page tri-fold questionnaire accompanied by a stamped, addressed return 
envelope; and (3) those sent the one-page questionnaire constructed as a self-mailer. 
The three questionnaire formats were sequentially assigned to graduates on lists from 
each college of the university* Since the questionnaires and return envelopes were 
identifiable only by college, responses were anonymous and there was no follow-up 
mailing. 

Overall response rates were contrasted using the hypothesis test of no 
significant difference in the proportions of mailed questionnaires returned under the 
three survey conditions. Responses to the scaled questionnaire items were contrasted 
across the three groups to determine if the groups receiving the three questionnaire 
formats exhibited differences in their opinions about the university. These 
hypotheses were tested at the ,05 significance level. Finally, the cost pe* viable 
returned questionnaire was compared across the three questionnaire formats. 

Results 

For the 7078 questionnaires mailed out, 1830 were returned for a 25.7% 
response rate. Response percentages across the three questionnaire formats were: 
(1) 27.9% for the two-page questionnaire with return envelope; (2) 26.3% for the one- 
page questionnaire with return envelope; and (3) 23.3% for the one-page questionnaire 
sent as a self-mailer. As shown in Table 1, these response rates were similar across 
colleges. For the eight colleges, the percentage returns ranged from 22.5% for Fine 
Arts to 28.5% for Communications. 

No significant difference was found between the proportions responding to 
formats (1) and (2), which both used a return envelope (z= 1.274, p>.05). However, 
a significantly higher proportion responded to format (1) having the two-page 
questionnaire than to format (3), the one-page self-mailer (z=3.633, p < .05). Similarly, 
a significantly higher proportion responded to format (2), the one-page questionnaire 
with return envelope, than to format (3), the one-page self-mailer (z=2.360, p<.05). 
Thus, significantly higher proportions of questionnaires were returned for formats 
using an enclosed return envelope than for the format which used a self-mailer. 

Average responses for the 20 questionnaire items ranged from 1.86 to 2.71 
on the four-point scale (l=excellent, 2= good, 3=fair and 4=poor). Three averages 
were in the 1.75-1.99 range, seven were in the 2.00 to 2.24 range, seven were in the 
2.25-2.49 range, and three were in the 2.50-2.74 range. Thus, 17 of the 20 items were 
rated in the "good* range (1.50 to 2.50) by the university graduates. 

In contrasting student responses across the three questionnaire formats, only 
one significant difference was noted among the 20 questionnaire items. As shown in 
Table 2, the Office of Career Planning and Placement was rated more favorably by 
those receiving the two-page questionnaire than those receiving the one-page self- 
mailer. 
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Table 1 
Response Rate by College 



College 
Agriculture 



Arts & Sciences 



Business 



Communications 



Education 



Engineering 



Fine Arts 



Nursing 



TOTAL 





Questionnaire Format 








(2) 


(3) 


Total 




113 


113 


113 


339 


No. Returned 


27 


31 


26 


84 


% Returned 


(23.9) 


(27.4) 


(23.0) 


(24.8) 


Nin MjiiIpH 


394 


393 


393 


1180 


No. Returned 


114 


116 


89 


319 


% Returned 


(28.9) 


(29.5) 


(22.6) 


(27.0) 


No Mailed 


678 


677 


677 


2032 


No. Returned 


186 


169 


145 


500 


% Returned 


(27.4) 


(25.0) 


(21.4) 


(24.6) 


No Mailed 


174 


174 


174 


522 


No. Returned 


54 


49 


46 


149 


% Returned 


(31.0) 


(28.2) 


(26.4) 


(28.5) 


No. Mailed 


600 


600 


601 


1801 


No. Returned 


176 


157 


155 


488 


% Returned 


(29.3) 


(26.2) 


(25.8) 


(27.1) 


No. Mailed 


60 


60 


59 


179 


No. Returned 


18 


18 


16 


52 


% Returned 


(30.0) 


(30.0) 


(27.1) 


(29.1) 


No. Mailed 


100 


99 


99 


298 


No. Returned 


17 


26 


24 


67 


% Returned 


(17.0) 


(26.3) 


(24.2) 


(22.5) 


No. Mailed 


243 


242 


242 


727 


No. Returned 


68 


54 


49 


171 


% Returned 


(28.0) 


(22.3) 


(20.2) 


(23.5) 


No. Mailed 


2362 


2358 


2358 


7078 


No. Returned 


660 


620 


550 


1830 


% Returned 


(27.9) 


(26.3) 


(23.3) 


(25.7) 
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The total costs of the mailings are given in Table 3. These costs included the 
printing of the cover letters and questionnaires, and the postage for the mail out and 
for the return. The most efficient format in this study was the one-page questionnaire 
sent with a return envelope ($1,256 per return) in contrast to the one-page self-mailer 
($1,381 per return) and the two-page questionnaire sent with a return envelope 
($1,591 per return). There was no significant difference in response rates on format 
(2) and format (1), which both used a return envelope. 



Table 2 

Average Responses by Item for 

Three Questionnaire Formats 

Format 1: 2 pages stapled with return envelope 
Format 2: 1 page with return envelope 
Format 3: 1 page self-mailer 



Questionnaire Item 




X, 
1.84 


1.85 


F 


F-prob. 


1. 


Teaching in major 


1.90 


1.461 


.232 


2. 


Teaching in other 


2.18 


2.12 


2.17 


1.498 


.224 


3. 


Prepared to compete 


2.11 


2.04 


2.12 


2.213 


.110 


4. 


Course content 


1.97 


1.94 


2.03 


1.755 


.173 


5. 


Need for first job 


2.22 


2.14 


2.19 


.947 


.388 


6. 


Academic advising 


2.43 


2.38 


2.38 


.401 


.670 


7. 


Faculty interaction 


2.18 


2.11 


2.10 


1.738 


.176 


8. 


Think and express 


2.09 


2.02 


2.09 


2.011 


.134 


9. 


Dormitory 


2.69 


2.71 


2.75 


.338 


.713 


10. 


Student activities 


2.50 


2.43 


2.51 


2.152 


.117 


11. 


Library 


1.90 


1.89 


1.97 


1.916 


.148 


12. 


Computer access 


2.51 


2.50 


2.85 


.565 


.568 


13. 


Financial Aids 


2.56 


2.57 


2.62 


.717 


.488 


14. 


Admissions/Records 


2.37 


2.33 


2.38 


.694 


.500 


15. 


Career Planning 


2.41 


2.46 


2.55 


3.228* 


.040 


16. 


Campus Security 


2.41 


2.44 


2.45 


.311 


.733 


17. 


Kept informed 


2.24 


2.20 


2.26 


.891 


.411 


18. 


ASU grads on job 


2.15 


2.10 


2.13 


.886 


.412 


19. 


ASU service 


2.30 


2.28 


2.35 


1.338 


.263 


20. 


Overall experience 


2.29 


2.18 


2.32 


2.255 


.106 



* p < .05 
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TableS 



Printing and Mailing Costs for Three Questionnaire Formats 





2-page/ 


1-page/ 


1-page/ 






envelope 


envelope 


self-mail 






0} 






Total 


Cover Letters 


$145.00 


$145.00 


$145.00 


$435.00 


Questionnaires 


432.00 


178.00 


178.00 


788.00 


Postage Out 


262.18 


261.74 


261.74 


785.66 


Postage In 


203.05 


191,58 


169.26 


563.89 


Total 


$1,042.23 


$776.32 


$754.00 


$2,572.55 


Cost per Return 


$1,591 


$1,256 


$1,381 


$1,414 



Discussion 

Although the overall response rate of this survey of university graduates was 
low, Smith and Bers (1987) report this was not an unusual outcome. Significant 
differences were noted between the response rates of the three questionnaire formats, 
however, there may be limited practical significance of these findings with the low 
28%, 26% and 23% response rates. In retrospect, another factor may have contributed 
to these results. The cover letter sent to the university graduates was signed by the 
university president who had announced his retirement six months earlier. This may 
have adversely affected the response rate. 

It was hypothesized the self-mailer would make it easier for the graduate to 
respond, thus increasing the response rate. This certainly was not the case in this 
investigation where both the one-page and two-page questionnaire formats using a 
return envelope had higher return rates than the one-page questionnaire sent as a 
self-mailer. 
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